Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions

In its simplest formulation, reinforcement learning is based on the idea that if an action taken in a particular context is followed by a favorable outcome, then, in the same context, the tendency to produce that action should be strengthened, or reinforced. While reinforcement learning forms the basis of many current theories of basal ganglia (BG) function, these models do not incorporate distinct computational roles for signals that convey context, and those that convey what action an animal takes. Recent experiments in the songbird suggest that vocal-related BG circuitry receives two functionally distinct excitatory inputs. One input is from a cortical region that carries context information about the current “time” in the motor sequence. The other is an efference copy of motor commands from a separate cortical brain region that generates vocal variability during learning. Based on these findings, I propose here a general model of vertebrate BG function that combines context information with a distinct motor efference copy signal. The signals are integrated by a learning rule in which efference copy inputs gate the potentiation of context inputs (but not efference copy inputs) onto medium spiny neurons in response to a rewarded action. The hypothesis is described in terms of a circuit that implements the learning of visually guided saccades. The model makes testable predictions about the anatomical and functional properties of hypothesized context and efference copy inputs to the striatum from both thalamic and cortical sources

An Automated Procedure for Evaluating Song Imitation

Songbirds have emerged as an excellent model system to understand the neural basis of vocal and motor learning. Like humans, songbirds learn to imitate the vocalizations of their parents or other conspecific “tutors.” Young songbirds learn by comparing their own vocalizations to the memory of their tutor song, slowly improving until over the course of several weeks they can achieve an excellent imitation of the tutor. Because of the slow progression of vocal learning, and the large amounts of singing generated, automated algorithms for quantifying vocal imitation have become increasingly important for studying the mechanisms underlying this process. However, methodologies for quantifying song imitation are complicated by the highly variable songs of either juvenile birds or those that learn poorly because of experimental manipulations. Here we present a method for the evaluation of song imitation that incorporates two innovations: First, an automated procedure for selecting pupil song segments, and, second, a new algorithm, implemented in Matlab, for computing both song acoustic and sequence similarity. We tested our procedure using zebra finch song and determined a set of acoustic features for which the algorithm optimally differentiates between similar and non-similar songs.National Institutes of Health (U.S.) (R01 MH067105

Vocal Experimentation in the Juvenile Songbird Requires a Basal Ganglia Circuit

Songbirds learn their songs by trial-and-error experimentation, producing highly variable vocal output as juveniles. By comparing their own sounds to the song of a tutor, young songbirds gradually converge to a stable song that can be a remarkably good copy of the tutor song. Here we show that vocal variability in the learning songbird is induced by a basal-ganglia-related circuit, the output of which projects to the motor pathway via the lateral magnocellular nucleus of the nidopallium (LMAN). We found that pharmacological inactivation of LMAN dramatically reduced acoustic and sequence variability in the songs of juvenile zebra finches, doing so in a rapid and reversible manner. In addition, recordings from LMAN neurons projecting to the motor pathway revealed highly variable spiking activity across song renditions, showing that LMAN may act as a source of variability. Lastly, pharmacological blockade of synaptic inputs from LMAN to its target premotor area also reduced song variability. Our results establish that, in the juvenile songbird, the exploratory motor behavior required to learn a complex motor sequence is dependent on a dedicated neural circuit homologous to cortico-basal ganglia circuits in mammals

Control of Vocal and Respiratory Patterns in Birdsong: Dissection of Forebrain and Brainstem Mechanisms Using Temperature

Learned motor behaviors require descending forebrain control to be coordinated with midbrain and brainstem motor systems. In songbirds, such as the zebra finch, regular breathing is controlled by brainstem centers, but when the adult songbird begins to sing, its breathing becomes tightly coordinated with forebrain-controlled vocalizations. The periods of silence (gaps) between song syllables are typically filled with brief breaths, allowing the bird to sing uninterrupted for many seconds. While substantial progress has been made in identifying the brain areas and pathways involved in vocal and respiratory control, it is not understood how respiratory and vocal control is coordinated by forebrain motor circuits. Here we combine a recently developed technique for localized brain cooling, together with recordings of thoracic air sac pressure, to examine the role of cortical premotor nucleus HVC (proper name) in respiratory-vocal coordination. We found that HVC cooling, in addition to slowing all song timescales as previously reported, also increased the duration of expiratory pulses (EPs) and inspiratory pulses (IPs). Expiratory pulses, like song syllables, were stretched uniformly by HVC cooling, but most inspiratory pulses exhibited non-uniform stretch of pressure waveform such that the majority of stretch occurred late in the IP. Indeed, some IPs appeared to change duration by the earlier or later truncation of an underlying inspiratory event. These findings are consistent with the idea that during singing the temporal structure of EPs is under the direct control of forebrain circuits, whereas that of IPs can be strongly influenced by circuits downstream of HVC, likely in the brainstem. An analysis of the temporal jitter of respiratory and vocal structure suggests that IPs may be initiated by HVC at the end of each syllable and terminated by HVC immediately before the onset of the next syllable.United States. National Institutes of Health (Grant R01 DC009183)United States. National Institutes of Health (Grant R01 MH067105

A role for descending auditory cortical projections in songbird vocal learning

Many learned motor behaviors are acquired by comparing ongoing behavior with an internal representation of correct performance, rather than using an explicit external reward. For example, juvenile songbirds learn to sing by comparing their song with the memory of a tutor song. At present, the brain regions subserving song evaluation are not known. In this study, we report several findings suggesting that song evaluation involves an avian 'cortical' area previously shown to project to the dopaminergic midbrain and other downstream targets. We find that this ventral portion of the intermediate arcopallium (AIV) receives inputs from auditory cortical areas, and that lesions of AIV result in significant deficits in vocal learning. Additionally, AIV neurons exhibit fast responses to disruptive auditory feedback presented during singing, but not during nonsinging periods. Our findings suggest that auditory cortical areas may guide learning by transmitting song evaluation signals to the dopaminergic midbrain and/or other subcortical targets.National Institutes of Health (U.S.) (grant R01 MH067105)McGovern Institute for Brain Research at MIT (Internal funding

Unsupervised discovery of temporal sequences in high-dimensional datasets, with applications to neuroscience.

Identifying low-dimensional features that describe large-scale neural recordings is a major challenge in neuroscience. Repeated temporal patterns (sequences) are thought to be a salient feature of neural dynamics, but are not succinctly captured by traditional dimensionality reduction techniques. Here, we describe a software toolbox-called seqNMF-with new methods for extracting informative, non-redundant, sequences from high-dimensional neural data, testing the significance of these extracted patterns, and assessing the prevalence of sequential structure in data. We test these methods on simulated data under multiple noise conditions, and on several real neural and behavioral datas. In hippocampal data, seqNMF identifies neural sequences that match those calculated manually by reference to behavioral events. In songbird data, seqNMF discovers neural sequences in untutored birds that lack stereotyped songs. Thus, by identifying temporal structure directly from neural data, seqNMF enables dissection of complex neural circuits without relying on temporal references from stimuli or behavioral outputs

Low-Power Circuits for Brain–Machine Interfaces

This paper presents work on ultra-low-power circuits for brain–machine interfaces with applications for paralysis prosthetics, stroke, Parkinson’s disease, epilepsy, prosthetics for the blind, and experimental neuroscience systems. The circuits include a micropower neural amplifier with adaptive power biasing for use in multi-electrode arrays; an analog linear decoding and learning architecture for data compression; low-power radio-frequency (RF) impedance-modulation circuits for data telemetry that minimize power consumption of implanted systems in the body; a wireless link for efficient power transfer; mixed-signal system integration for efficiency, robustness, and programmability; and circuits for wireless stimulation of neurons with power-conserving sleep modes and awake modes. Experimental results from chips that have stimulated and recorded from neurons in the zebra finch brain and results from RF power-link, RF data-link, electrode- recording and electrode-stimulating systems are presented. Simulations of analog learning circuits that have successfully decoded prerecorded neural signals from a monkey brain are also presented

Natural Changes in Brain Temperature Underlie Variations in Song Tempo during a Mating Behavior

The song of a male zebra finch is a stereotyped motor sequence whose tempo varies with social context – whether or not the song is directed at a female bird – as well as with the time of day. The neural mechanisms underlying these changes in tempo are unknown. Here we show that brain temperature recorded in freely behaving male finches exhibits a global increase in response to the presentation of a female bird. This increase strongly correlates with, and largely explains, the faster tempo of songs directed at a female compared to songs produced in social isolation. Furthermore, we find that the observed diurnal variations in song tempo are also explained by natural variations in brain temperature. Our findings suggest that brain temperature is an important variable that can influence the dynamics of activity in neural circuits, as well as the temporal features of behaviors that some of these circuits generate